Image quality compensation system and method

ABSTRACT

A user authentication system and method. The user authentication system includes a camera and a processor connected to the camera. The processor receives images from the camera, searches for a user feature in the images, determines if the images require correction, adjusts camera controls in a pre-defined order to provide desired corrections, applies the desired corrections to subsequent images and authenticates the user based on the user feature in the corrected images.

PRIORITY APPLICATION

This application is a U.S. National Stage Application under 35 U.S.C.371 from International Application Number PCT/US2014/071527, filed Dec.19, 2014, which claims the benefit of priority to U.S. ProvisionalApplication Ser. No. 62/079,003, filed Nov. 13, 2014, all of which areincorporated herein by reference in their entirety.

BACKGROUND

Facial recognition in PCs, tablets and phones is very dependent on imagequality produced by integrated user-facing cameras, which in turn arevery dependent on ambient lighting conditions. These cameras areprimarily calibrated to tune their settings for best photographicquality as opposed to focusing on the user's face. As a result insub-optimal lighting conditions, e.g. dark room, direct sunlight, brightbackground (e.g., open window), the user's face is too dark, too brightor silhouetted. This severely hampers face detection and recognition,thus making facial recognition an unreliable experience across typicalusage environments.

Current facial recognition solutions suffer from degraded performanceand, in some cases, fail to even detect the user's face in adverselighting conditions, such as when the illumination is less than 30 lux(dark areas), greater than 10,000 lux (direct sunlight) or obscured by abright background (e.g., when the foreground illumination is less than100 lux and the background is greater than 1000 lux). Webcams performunevenly across platforms and OEMs and are not calibrated to highlightthe face over the background in adverse lighting conditions, leading tothe failure of face recognition.

A common technique used to compensate for low light is to convert thelaptop/tablet screen to an all-white image, thus using the screen'sbrightness to illuminate the subject (e.g. Sensible Vision'sFaceBright). However, tablet/notebook screen brightness is frequentlyauto-tuned by ambient light sensors to a low setting in low lightingconditions, resulting in the white screen not being very bright andhence hampering the efficacy of this method. Distance of face from thecomputer screen is another factor contributing to this method'squestionable reliability. Also this method does not address the problemsinherent from direct sunlight and from bright backgrounds.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are illustrated by way of example, and not by way oflimitation, in the figures of the accompanying drawings in which likereference numerals refer to similar elements.

FIGS. 1A and 1B illustrate feature-based authentication systems;

FIGS. 2A and 2B illustrate a method for correcting an image to be usedin the authentication system of FIG. 1;

FIG. 3A shows a face image taken against a bright background;

FIG. 3B illustrates correction of the image shown in FIG. 3A;

FIG. 4A shows a face image taken in low light conditions;

FIG. 4B illustrates correction of the image shown in FIG. 4A;

FIG. 5A shows a face image taken in very low light conditions;

FIG. 5B illustrates correction of the image shown in FIG. 5A; and

FIG. 6 is a block diagram illustrating an example machine upon which anyone or more of the techniques (e.g., methodologies) discussed herein mayperform, according to an example embodiment.

DESCRIPTION OF THE EMBODIMENTS

As noted above, facial recognition in PCs, tablets and phones is verydependent on image quality produced by integrated user-facing cameras,which in turn are very dependent on ambient lighting conditions. Thesecameras are primarily calibrated to tune their settings for bestphotographic quality as opposed to focusing on the user's face. As aresult, in sub-optimal lighting conditions, the image captured is lessthan optimal. This severely hampers face detection and recognition, thusmaking facial recognition an unreliable experience across typical usageenvironments.

FIG. 1A illustrates a feature-based authentication system 100. System100 of FIG. 1A includes a computing system 102 connected to one or morecameras 104 via a connector 106. In some embodiments, computer system102 includes a display 108, a processor and memory used to store dataand programs. In some embodiments, an input device 110 (such as, forexample, a keyboard or a mouse) is connected to computing system 102 aswell. In some embodiments, connector 106 is a Universal Serial Bus (USB)cable, while in other embodiments connector 106 is a network such asEthernet.

In one embodiment, system 100 aims to resolve the reliability problemnoted above by overriding the firmware of camera 104 and employing asystem to externally adjust camera settings such as exposure, gammacorrection and gain to compensate for ambient lighting conditions. Inone such embodiment, the goal is to homogenize facial recognitionefficacy and reliability across all lighting conditions and cameras.

In one embodiment, system 100 resolves the issue of sub-optimal lightingconditions by tuning camera image capture settings to highlight the faceacross all lighting conditions, including the use cases cited above,thus enabling optimal face detection and recognition no matter where theuser is, the system they are using or the type of camera 104 in theirsystem 100.

In one example embodiment, a sub-optimal lighting image compensationsystem operating in either camera 104 or computer system 102 overridesdefault camera firmware behavior using video control and camera controlinterfaces, namely exposure, gamma correction and gain control to adjustimage quality for optimal face detection. In one such embodiment, thesystem runs in a feedback loop running face detection and checking imagequality in real time using Intel Integrated Performance Primitives (IPP)image processing libraries. The Intel Integrated Performance Primitives(IPP) image processing libraries are libraries of highly optimizedbuilding blocks for media and data applications. In one embodiment,camera adjustments are made based on image quality feedback receivedafter each image processing iteration, specifically targeting enhancingface visibility over general photographic quality. The result is animage with enhanced sharpness and signal-to-noise ratio (SNR) that aidsedge detection and feature extraction tailored to enable face detectionin virtually any lighting condition.

Authentication system 100 can be implemented in any camera enabledcompute system, including desktops, laptops, tablets and phones. Oneexample embodiment of a laptop-based authentication system 100 is shownin FIG. 1B. System 100 of FIG. 1B includes a laptop system 112 with oneor more internal cameras 104. In some embodiments, laptop system 112includes a display 108, a processor and memory used to store data andprograms. In some embodiments, an input device 110 (such as, forexample, a keyboard or a mouse) is connected to laptop system 112 aswell.

As noted above in the discussion of FIG. 1A, system 100 of FIG. 1B, inone embodiment, aims to resolve the reliability problem noted above byoverriding the firmware of integrated camera 104 and employing a systemto externally adjust camera settings such as exposure, gamma correctionand gain to compensate for ambient lighting conditions. In one suchembodiment, the goal is to homogenize facial recognition efficacy andreliability across all lighting conditions and cameras. In some suchembodiments, laptop system 112 executes program code in a feedback looprunning face detection and checking image quality in real time usingIntel Integrated Performance Primitives (IPP) image processinglibraries.

A method of adjusting camera 104 to provide an optimized image forfacial recognition is shown in FIGS. 2A and 2B. At 200, a new frame iscaptured by camera 104. At 202, the current camera control settings forcamera 104 are saved to memory (as camera state S0). At 204, thecaptured frame is processed to detect a face. In the following, a faceimage will be used to demonstrate the method shown in FIGS. 2A and 2B.Other user features, such as feet, hands, or the shape of the user'seyes, could be used as well.

In one embodiment, a score is calculated at 204 that is a measure of thequality of the face image. In one such embodiment face image quality isa function of analysis of facial landmarks in the image. In oneembodiment, face detection and quality scoring is performed with IPPprimitives.

At 206, if no face has been detected in the frame or the face score isless than a threshold value for the last three consecutive frames,control moves to 210. Otherwise, control moves to 208, the frame is sentto the facial recognition application and the state of the cameracontrols is set to its state S0.

At 210, an image quality check is conducted on the frame. In oneembodiment, the reason that the method reached 210 is because the imageis either too bright or too dark to obtain a good image of the face. Insuch an embodiment, the image quality check at 210 determines if theimage is too dark (due to, for example, low light) or too bright (dueto, for example, bright sunlight). If too dark, control moves to 212 andthe camera controls are set to a dark image correction mode. If toobright, control moves to 214 and the camera controls are set to a brightimage correction mode (for instance, to reduce exposure). Control thenmoves to 216.

In some embodiments, the image quality check is done on either the faceimage, or on the center 40% of the full frame if no face is available.In one such embodiment, light is measured using a gray scale histogramof the portion of the image being reviewed.

At 216 a check is made to determine if system 100 is still correctingthe image for the image correction mode (dark or bright imagecorrection) that was first determined at 210. As noted above, thequality of the image is checked at 210 to determine if we need toimprove a dark (low light) image or a bright (overexposed) image. If theimage correction mode has changed from the previous image correctionmode (for example, a bright light was turned on in the dimly lit room,or a bright light was turned off in what became a dimly lit room), thenthe previous corrections are no longer viable. In the embodiment shownin FIG. 2A, control then moves to 218 and camera controls settings forcamera 104 are restored to state S0.

In some embodiments, at 218, camera control settings for camera 104transition gradually back to state S0 over a predefined number offrames.

If, at 216, it is determined that system 100 is still correcting theimage for the image correction mode (dark or bright image correction)that was first determined at 210, further image correction is needed.Control moves to Sx, where x is the current state of the camera controlsand, in the embodiment shown in FIGS. 2A and 2B, is an integer between 1and 4.

If x=0, control moves to S0, and all camera controls are set to manualmode at 220. In the embodiment shown in FIG. 2B, this includes exposure,gain and gamma. Control them moves to S1.

At S1, control moves to 222 and a check is made to see if there iseither exposure correction or backlight compensation in camera 104. Ifnot, control moves to S2 with the current state of camera controls setto S2.

If, however, either exposure correction or backlight compensation existsin camera 104, control moves to 224 and, if in dark image mode of thisembodiment, backlight compensation is increased and exposure isincreased. If, at 224, the image mode is bright image mode, backlightcompensation is increased and exposure is reduced.

Control then moves to 226, where a check is made to see if exposurecorrection and/or backlight compensation is at its limits. If not,control moves to 204 with the current state of camera controls set to S1and the current camera controls are applied to the next image receivedfrom camera 104.

If, however, exposure correction and/or backlight compensation is at itslimits, camera control state is set to S2 at 228 before control moves to204.

At S2, control moves to 230 and a check is made to see if there is gammacorrection in camera 104. If not, control moves to S3 with the currentstate of camera controls set to S3.

If, however, the check at 230 determines that there is gamma correctionin camera 104, control moves to 232. At 232, if the image mode is indark image correction mode, gamma correction is increased. In oneembodiment, the correction is a 10% step. At 232, if the image mode isin bright image correction mode, gamma correction is decreased. In oneembodiment, the decrease is a 10% step. Control then moves to 234, wherea check is made to see if gamma correction is nearing its limits If not,control moves to 204 with the current state of camera controls set toS2. The current camera controls are then applied to the next imagereceived from camera 104.

If, however, gamma correction is nearing its limits at 234, cameracontrol state is set to S3 at 236 before control moves to 204.

At S3, control moves to 238 and a check is made to see if gaincorrection is available in camera 104. If not, control moves to S4 withthe current state of camera controls set to S4. The current S4 cameracontrols are then applied to the next image received from camera 104.

If, however, the check at 238 determines that there is gain correctionin camera 104, control moves to 240. At 240, if the image mode is indark image correction mode, gain correction is increased. In oneembodiment, the correction is a 10% step. At 240, if the image mode isin bright image correction mode, gain correction is decreased. In oneembodiment, the decrease is a 10% step. Control then moves to 242, wherea check is made to see if gain correction is nearing its limits. If not,control moves to 204 with the current state of camera controls set toS3. The current camera controls are then applied to the next imagereceived from camera 104.

If, however, gain correction is nearing its limits at 242, cameracontrol state is set to S4 at 244 before control moves to 204.

In some embodiments, a determination is made at 216 whether the imagerequires additional correction, whether the image was overcorrected orwhether a new frame illustrates that lighting conditions have changedsince the last iteration. In some such embodiments, if system 100overshoots on its correction, system 100 reduces the correction by halfthe amount system 100 uses to increase correction.

The solution described above works on PCs, tablets and phones as it isindependent of screen lighting and robust against motion (i.e., the userwalking with system in hand) The use of Intel Integrated PerformancePrimitives libraries helps system 100 perform image compensation inreal-time (that is, at frame capture rate) thus ensuring adequateperformance across a variety of platforms.

Our approach differs from existing consumer-based facial resolutionapplications primarily by targeting image quality compensation toenhance computer vision-based face detection (i.e., photographic qualityis sacrificed to produce an image that may look noisy andcolor-challenged to the human eye but is optimal for machine learningalgorithms to detect a face and extract features). In one embodiment,system 100 overrides the camera's normal exposure, gamma and gainsettings to arrive at the best face image possible (for facialrecognition) given the existing lighting conditions.

Other camera controls may be available as well, such as brightness,contrast, white balance, sharpness, saturation and focus.

Examples of application of the above methods are provided in FIGS.3B-5B. In the pictures shown in FIGS. 3A-5A, the image is one that wascaptured by a camera 104 and which failed face detection. Thecorresponding images in FIGS. 3B-5B, show the compensated image withsuccessful face detection. In the example embodiments shown in FIGS.3B-5B, the green brackets indicate that a face of sufficient quality forfacial recognition was found. Subsequent tests confirmed that thecompensated images were successfully matched for the user, resulting inauthentication of the user to the device.

In the example shown in FIG. 3A, the image is one taken with a brightbackground. In the example the picture was taken with a window behindthe subject. The subject's face is underexposed. Application of themethods above made the face more recognizable and susceptible for facialrecognition programs, as can be seen in the image of FIG. 3B. In someembodiments, if system 100 identifies what appears to be a face, itallows subsequent frames to be overexposed as necessary to reveal facialfeatures required for authentication.

In the example shown in FIG. 4A, the picture was taken in low lightconditions. Application of the methods discussed above, highlight thesubject's face to the detriment of any background fidelity as can beseen in FIG. 4B.

Finally, in the example shown in FIG. 5A, the picture was taken in avery low light environment. In one example embodiment, in a dark room,camera 104 adjusts camera settings in different ways to attempt to findedges that represent the outline of a face. In a very dark room, in someembodiments, camera 104 adjusts exposure to be very high, resulting in anoisy image as can be seen in FIG. 5B.

In one such embodiment, if no face is detected, system 100 assumes aface will be found in the center of the image. In such embodiments, thecenter of the image is the region of interest (ROI). In some suchembodiments, the ROI is the center 40% of the full frame; system 100measures image intensity across the ROI and raises gamma, exposure andgain as necessary to enhance facial features for authentication.

As can be seen in FIG. 5B, the corrections above were able to capturefacial characteristics at the price of including some artifacts in thecorrected picture. As noted above, the methods described above targetenhancing face visibility over general photographic quality. The resultis an image with enhanced sharpness and signal-to-noise ratio (SNR) thataids edge detection and feature extraction tailored to enable facedetection in virtually any lighting condition.

It is worth noting that the method described above does not rely on theavailability of each of the three control methods discussed in thecontext of FIGS. 2A and 2B. Instead, the method is functional even ifonly one or a couple of these controls are available on a given camera104. This enables the approach to support the widest range of systemsexpected to be available in the market. The above approach can also beused with other camera controls to extract face images from poorly litimages.

In addition, it is not necessary for the processor performing imagecompensation to also perform the authentication. In some embodiments, aprocessor in camera 104 performs the image compensation while aprocessor in the computing device performs the authentication based onthe image corrected images. In other embodiments, image compensation isperformed on the computing device and authentication based on the imagecorrected images is performed on a server, or in the cloud. In yet otherembodiments, image compensation is performed on the camera andauthentication based on the corrected images is performed on a server,or in the cloud.

The same can also be true for the invention itself, for best real-timeperformance we run it on the same processor connected to the camera, butit can potentially be run in the cloud as well, the real timeperformance will suffer but it would still work. Should probably addthis to the claims that the processor does not necessarily need to beconnected to the camera.

FIG. 6 is a block diagram illustrating a machine in the example form ofa computer system 102, within which a set or sequence of instructionsmay be executed to cause the machine to perform any one of themethodologies discussed herein, according to an example embodiment. Inalternative embodiments, the machine operates as a standalone device ormay be connected (e.g., networked) to other machines. In a networkeddeployment, the machine may operate in the capacity of either a serveror a client machine in server-client network environments, or it may actas a peer machine in peer-to-peer (or distributed) network environments.The machine may be a personal computer (PC), a tablet PC, a hybridtablet, a set-top box (STB), a personal digital assistant (PDA), amobile telephone, a web appliance, a network router, switch or bridge,or any machine capable of executing instructions (sequential orotherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute a set (or multiple sets) of instructions to perform anyone or more of the methodologies discussed herein.

Example computer system 102 includes at least one processor 1002 (e.g.,a central processing unit (CPU), a graphics processing unit (GPU) orboth, processor cores, compute nodes, etc.), a main memory 1004 and astatic memory 1006, which communicate with each other via a link 1008(e.g., bus). The computer system 102 may further include a video displayunit 1010, an alphanumeric input device 1012 (e.g., a keyboard), and auser interface (UI) navigation device 1014 (e.g., a mouse). In oneembodiment, the video display unit 1010, input device 1012 and UInavigation device 1014 are incorporated into a touch screen display. Thecomputer system 102 may additionally include a storage device 1016(e.g., a drive unit), a signal generation device 1018 (e.g., a speaker),a network interface device 1020, and one or more sensors (not shown),such as a global positioning system (GPS) sensor, compass,accelerometer, or other sensor.

The storage device 1016 includes a machine-readable medium 1022 on whichis stored one or more sets of data structures and instructions 1024(e.g., software) embodying or utilized by any one or more of themethodologies or functions described herein. The instructions 1024 mayalso reside, completely or at least partially, within the main memory1004, static memory 1006, and/or within the processor 1002 duringexecution thereof by the computer system 102, with the main memory 1004,static memory 1006, and the processor 1002 also constitutingmachine-readable media.

While the machine-readable medium 1022 is illustrated in an exampleembodiment to be a single medium, the term “machine-readable medium” mayinclude a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more instructions 1024. The term “machine-readable medium”shall also be taken to include any tangible medium that is capable ofstoring, encoding or carrying instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present disclosure or that is capable of storing,encoding or carrying data structures utilized by or associated with suchinstructions. The term “machine-readable medium” shall accordingly betaken to include, but not be limited to, solid-state memories, andoptical and magnetic media. Specific examples of machine-readable mediainclude non-volatile memory, including, but not limited to, by way ofexample, semiconductor memory devices (e.g., electrically programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory

(EEPROM)) and flash memory devices; magnetic disks such as internal harddisks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks.

The instructions 1024 may further be transmitted or received over acommunications network 1026 using a transmission medium via the networkinterface device 1020 utilizing any one of a number of well-knowntransfer protocols (e.g., HTTP). Examples of communication networksinclude a local area network (LAN), a wide area network (WAN), theInternet, mobile telephone networks, plain old telephone (POTS)networks, and wireless data networks (e.g., Wi-Fi, 3G, and 4G LTE/LTE-Aor WiMAX networks). The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding, orcarrying instructions for execution by the machine, and includes digitalor analog communications signals or other intangible medium tofacilitate communication of such software.

Additional Notes & Examples:

Example 1 includes subject matter for a user authentication system (suchas a device, apparatus, or machine) comprising: a camera; and aprocessor, connected to the camera, wherein the processor receivesimages from the camera, searches for a user feature in the images,determines if the images require correction, adjusts camera controls ina pre-defined order to provide desired corrections, applies the desiredcorrections to subsequent images and authenticates the user based on theuser feature in the corrected images.

In Example 2, the subject matter of Example 1 may include, wherein theimages are video frames.

In Example 3, the subject matter of any one of Examples 1 to 2 mayinclude, wherein the processor adjusts camera controls for exposure.

In Example 4, the subject matter of any one of Examples 1 to 3 mayinclude, wherein the processor adjusts camera controls for gain.

In Example 5, the subject matter of any one of Examples 1 to 4 mayinclude, wherein the processor adjusts camera controls for gamma.

In Example 6, the subject matter of any one of Examples 1 to 5 mayinclude, wherein the processor adjusts camera controls selected from thegroup of camera controls consisting of camera controls for exposure, forgamma and for gain.

In Example 7, the subject matter of any one of Examples 1 to 6 mayinclude, wherein the user feature is a face.

In Example 8, the subject matter of any one of Examples 1 to 7 mayinclude, wherein the processor determines when subsequent images nolonger need correction and sets the camera controls to a differentconfiguration.

In Example 9, the subject matter of any one of Examples 1 to 8 mayinclude, wherein the processor determines when subsequent images nolonger need correction and sets the camera controls to an initial state.

In Example 10, the subject matter of any one of Examples 1 to 9 mayinclude, wherein the processor sets the camera controls back to aninitial state over a plurality of frames when subsequent images nolonger need correction.

In Example 11, the subject matter of any one of Examples 1 to 10 mayinclude, wherein the camera is separate from the processor.

In Example 12, the subject matter of any one of Examples 1 to 11 mayinclude, wherein the processor is in a computing device and wherein thecamera is incorporated into the computing device.

In Example 13, the subject matter of any one of Examples 1 to 12 mayinclude, wherein the processor determines if the image requirescorrection by checking image quality.

In Example 14, the subject matter of any one of Examples 1 to 13 mayinclude, wherein the processor determines if the image requirescorrection by analyzing facial landmarks in the image.

In Example 15, the subject matter of any one of Examples 1 to 14 mayinclude, wherein the camera includes one or more of a visual lightsensor and an infrared sensor.

Example 16 includes subject matter (such as a method, means forperforming acts, machine readable medium including instructions thatwhen performed by a machine cause the machine to performs acts, or anapparatus to perform) comprising: capturing images of a user, whereineach image includes a user feature; searching for the user feature inthe captured image; determining if the image requires correction; if theimage requires correction, adjusting camera controls in a pre-definedorder to provide desired corrections; and applying the corrections tosubsequent images.

In Example 17, the subject matter of Example 16 may include, whereincapturing images includes extracting the images from video.

In Example 18, the subject matter of any one of Examples 16 to 17 mayinclude, wherein adjusting camera controls includes adjusting cameracontrols for exposure.

In Example 19, the subject matter of any one of Examples 16 to 18 mayinclude, wherein adjusting camera controls includes adjusting cameracontrols for gain.

In Example 20, the subject matter of any one of Examples 16 to 19 mayinclude, wherein adjusting camera controls includes adjusting cameracontrols for gamma.

In Example 21, the subject matter of any one of Examples 16 to 20 mayinclude, wherein adjusting camera controls includes selecting a cameracontrol from the group of camera controls consisting of camera controlsfor exposure, camera controls for gamma and camera controls for gain.

In Example 22, the subject matter of any one of Examples 16 to 21 mayinclude, wherein adjusting camera controls includes: a) adjusting cameracontrols for exposure; b) checking image quality of one or moresubsequent images; c) if further correction is needed, adjusting cameracontrols for gamma; d) checking image quality of one or more subsequentimages; and e) if further correction is needed, adjusting cameracontrols for gain.

In Example 23, the subject matter of any one of Examples 16 to 22 mayinclude, wherein searching for the user feature in the captured imageincludes detecting a face.

In Example 24, the subject matter of any one of Examples 16 to 23 mayinclude, wherein the method further comprises authenticating the userbased on the user feature in the corrected images.

In Example 25, the subject matter of any one of Examples 16 to 24 mayinclude, wherein authenticating includes executing a cloud-basedauthentication routine.

In Example 26, the subject matter of any one of Examples 16 to 25 mayinclude, wherein the method further comprises: determining whensubsequent images no longer need correction; and resetting the cameracontrols to an initial state when subsequent images no longer needcorrection.

In Example 27, the subject matter of any one of Examples 16 to 26 mayinclude, wherein the correction includes bright image correction.

In Example 28, the subject matter of any one of Examples 16 to 27 mayinclude, wherein the correction includes dark image correction.

In Example 29, the subject matter of any one of Examples 16 to 28 mayinclude, wherein determining if the image requires correction includesexecuting program code in a processor to check image quality.

In Example 30, the subject matter of any one of Examples 16 to 29 mayinclude, wherein determining if the image requires correction includeschecking image quality.

In Example 31, the subject matter of any one of Examples 16 to 30 mayinclude, wherein checking image quality includes generating a grayscalehistogram of the image.

In Example 32, the subject matter of any one of Examples 16 to 31 mayinclude, wherein determining if the image requires correction includesanalyzing facial landmarks in the image.

In Example 33, the subject matter of any one of Examples 16 to 32 mayinclude, wherein determining if the image requires correction includesdetecting a face, measuring image quality of the face detected andcorrecting the image as a function of the measured image quality.

Example 34 includes at least one machine-readable medium includinginstructions, which when executed by a machine, cause the machine toperform operations of any of the Examples 16-33.

Example 35 includes an apparatus comprising means for performing any ofthe Examples 16-33.

Example 36 includes subject matter (such as a device, apparatus, ormachine) comprising: a camera; and a processor, connected to the camera,wherein the processor includes: means for searching for a user featurein an image received from the camera; means for determining if the imagerequires correction; and means for adjusting camera controls in apre-defined order to provide desired corrections.

In Example 37, the subject matter of Example 36 may include, wherein themeans for searching for a user feature includes means for extracting theimage from a video frame.

In Example 38, the subject matter of any one of Examples 36 to 37 mayinclude, wherein the means for adjusting camera controls includes meansfor adjusting camera controls for exposure.

In Example 39, the subject matter of any one of Examples 36 to 38 mayinclude, wherein the means for adjusting camera controls includes meansfor adjusting camera controls for gain.

In Example 40, the subject matter of any one of Examples 36 to 39 mayinclude, wherein the means for adjusting camera controls includes meansfor adjusting camera controls for gamma.

In Example 41, the subject matter of any one of Examples 36 to 40 mayinclude, wherein the means for adjusting camera controls includes meansfor adjusting exposure, gamma and gain.

In Example 42, the subject matter of any one of Examples 36 to 41 mayinclude, wherein the means for searching for the user feature in theimage includes means for detecting a face.

In Example 43, the subject matter of any one of Examples 36 to 42 mayinclude, wherein the apparatus further includes means for authenticatinga user as a function of the user feature.

In Example 44, the subject matter of any one of Examples 36 to 43 mayinclude, wherein the processor further includes: means for determiningwhen subsequent images no longer need correction; and means forresetting the camera controls to an initial state when subsequent imagesno longer need correction.

In Example 45, the subject matter of any one of Examples 36 to 44 mayinclude, wherein the correction includes bright image correction.

In Example 46, the subject matter of any one of Examples 36 to 45 mayinclude, wherein the correction includes dark image correction.

In Example 47, the subject matter of any one of Examples 36 to 46 mayinclude, wherein the means for determining if the image requirescorrection includes means for executing program code in a processor tocheck image quality.

In Example 48, the subject matter of any one of Examples 36 to 47 mayinclude, wherein the means for determining if the image requirescorrection includes means for checking image quality.

In Example 49, the subject matter of any one of Examples 36 to 48 mayinclude, wherein the means for checking image quality includes means forgenerating a grayscale histogram of the image.

In Example 50, the subject matter of any one of Examples 36 to 49 mayinclude, wherein the means for determining if the image requirescorrection includes means for analyzing facial landmarks in the image.

In Example 51, the subject matter of any one of Examples 36 to 50 mayinclude, wherein the means for determining if the image requirescorrection includes means for detecting a face, means for measuringimage quality of the detected face and means for determining, as afunction of the measured image quality of the detected face, cameracontrols to adjust.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments that may bepracticed. These embodiments are also referred to herein as “examples.”Such examples may include elements in addition to those shown ordescribed. However, also contemplated are examples that include theelements shown or described. Moreover, also contemplate are examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

Publications, patents, and patent documents referred to in this documentare incorporated by reference herein in their entirety, as thoughindividually incorporated by reference. In the event of inconsistentusages between this document and those documents so incorporated byreference, the usage in the incorporated reference(s) are supplementaryto that of this document; for irreconcilable inconsistencies, the usagein this document controls.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” includes “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In the appended claims, the terms “including” and“in which” are used as the plain-English equivalents of the respectiveterms “comprising” and “wherein.” Also, in the following claims, theterms “including” and “comprising” are open-ended, that is, a system,device, article, or process that includes elements in addition to thoselisted after such a term in a claim are still deemed to fall within thescope of that claim. Moreover, in the following claims, the terms“first,” “second,” and “third,” etc. are used merely as labels, and arenot intended to suggest a numerical order for their objects.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) may be used in combination with others. Otherembodiments may be used, such as by one of ordinary skill in the artupon reviewing the above description. The Abstract is to allow thereader to quickly ascertain the nature of the technical disclosure, forexample, to comply with 37 C.F.R. §1.72(b) in the United States ofAmerica. It is submitted with the understanding that it will not be usedto interpret or limit the scope or meaning of the claims. Also, in theabove Detailed Description, various features may be grouped together tostreamline the disclosure. However, the claims may not set forth everyfeature disclosed herein as embodiments may feature a subset of saidfeatures. Further, embodiments may include fewer features than thosedisclosed in a particular example. Thus, the following claims are herebyincorporated into the Detailed Description, with a claim standing on itsown as a separate embodiment. The scope of the embodiments disclosedherein is to be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

The invention claimed is:
 1. A user authentication system, comprising: acamera; and a processor, connected to the camera, wherein the processorreceives images from the camera, searches for a user feature in theimages, determines if the images require correction, adjusts cameracontrols in a pre-defined order to provide desired corrections, appliesthe desired corrections to subsequent images and authenticates the userbased on the user feature in the corrected images, wherein to adjustcamera controls, the processor: a) adjusts camera controls for exposure;b) checks image quality of one or more subsequent images; c) if furthercorrection is needed, adjusts camera controls for gamma; d) checks imagequality of one or more subsequent images; and e) if further correctionis needed, adjusts camera controls for gain.
 2. The system of claim 1,wherein the processor adjusts camera controls selected from the group ofcamera controls consisting of camera controls for exposure, for gammaand for gain.
 3. The system of claim 1, wherein the processor determineswhen subsequent images no longer need correction and sets the cameracontrols to a different configuration.
 4. The system of claim 1, whereinthe processor determines when subsequent images no longer needcorrection and sets the camera controls to an initial state.
 5. Thesystem claim 1, wherein the camera includes one or more of a visuallight sensor and an infrared sensor.
 6. A method, comprising: capturingimages of a user, wherein each image includes a user feature; searchingfor the user feature in the captured image; determining if the imagerequires correction; if the image requires correction, adjusting cameracontrols in a pre-defined order to provide desired corrections; andapplying the corrections to subsequent images, wherein adjusting cameracontrols includes: a) adjusting camera controls for exposure; b)checking image quality of one or more subsequent images; c) if furthercorrection is needed, adjusting camera controls for gamma; d) checkingimage quality of one or more subsequent images; and e) if furthercorrection is needed, adjusting camera controls for gain.
 7. The methodof claim 6, wherein adjusting camera controls includes adjusting cameracontrols for exposure.
 8. The method of claim 6, wherein adjustingcamera controls includes adjusting camera controls for gain.
 9. Themethod of claim 6, wherein adjusting camera controls includes adjustingcamera controls for gamma.
 10. The method of claim 6, wherein adjustingcamera controls includes selecting a camera control from the group ofcamera controls consisting of camera controls for exposure, cameracontrols for gamma and camera controls for gain.
 11. The method of claim6, wherein searching for the user feature in the captured image includesdetecting a face.
 12. The method of claim 6, wherein the method furthercomprises authenticating the user based on the user feature in thecorrected images.
 13. The method of claim 6, wherein the method furthercomprises: determining when subsequent images no longer need correction;and resetting the camera controls to an initial state when subsequentimages no longer need correction.
 14. The method of claim 6, wherein thecorrection includes dark image correction.
 15. The method of claim 6,wherein determining if the image requires correction includes checkingimage quality.
 16. The method of claim 15, wherein checking imagequality includes generating a grayscale histogram of the image.
 17. Themethod of claim 6, wherein determining if the image requires correctionincludes analyzing facial landmarks in the image.
 18. At least onenon-transitory machine-readable medium including instructions, whichwhen executed by a machine, cause the machine to: capture images of auser, wherein each image includes a user feature; search for the userfeature in the captured image; determine if the image requirescorrection; if the image requires correction, adjust camera controls ina pre-defined order to provide desired corrections; and apply thecorrections to subsequent images, wherein the instructions to adjustcamera controls include instructions to: a) adjust camera controls forexposure; b) check image quality of one or more subsequent images; c) iffurther correction is needed, adjust camera controls for gamma; d) checkimage quality of one or more subsequent images; and e) if furthercorrection is needed, adjust camera controls for gain.
 19. The at leastone non-transitory machine-readable medium of claim 18, wherein theinstructions to adjust camera controls include instructions to adjustcamera controls for exposure.
 20. The at least one non-transitorymachine-readable medium of claim 18, wherein instructions to search forthe user feature in the captured image include instructions to detect aface.
 21. The at least one non-transitory machine-readable medium ofclaim 18, comprising instructions to: determine when subsequent imagesno longer need correction; and reset the camera controls to an initialstate when subsequent images no longer need correction.
 22. The at leastone non-transitory machine-readable medium of claim 18, wherein theinstructions to determine if the image requires correction includeinstructions to checking image quality.
 23. The at least onenon-transitory machine-readable medium of claim 22, wherein theinstructions to check image quality include instructions to generate agrayscale histogram of the image.
 24. The at least one non-transitorymachine-readable medium of claim 18, wherein the instructions todetermine if the image requires correction include instructions toanalyze facial landmarks in the image.